DataFrame Sort & OrderBy Operations

Sorting rows in a DataFrame based on one or more columns in ascending or descending order.

What are the Sort and OrderBy Operations?

The sort() and orderBy() operations sort rows in a DataFrame by the values of specified columns, equivalent to the ORDER BY clause in standard ANSI SQL.

In PySpark, sort() and orderBy() are completely identical aliases—they share the same underlying logical execution plans.

Syntax and Ascending/Descending Direction

By default, PySpark sorts in ascending order. To sort in descending order, wrap the column references using desc() or .desc().

from pyspark.sql.functions import col, desc

# A. Sort ascending by single column
df.orderBy("age")

# B. Sort descending using col object
df.sort(col("salary").desc())

# C. Multi-column sorting (e.g. Department ascending, then Salary descending)
df.orderBy(col("department").asc(), col("salary").desc())

Example Usage Pipeline

Below is a complete, copy-paste-ready PySpark script demonstrating sorting operations:

from pyspark.sql import SparkSession
from pyspark.sql.functions import col

# 1. Setup local Spark session
spark = SparkSession.builder \
    .appName("DataFrame Sorting Demo") \
    .master("local[*]") \
    .getOrCreate()

# 2. Dummy dataset (Products)
data = [
    ("Laptop", "Electronics", 1200.0, 4.8),
    ("Headphones", "Electronics", 150.0, 4.2),
    ("Smartphone", "Electronics", 800.0, 4.6),
    ("Running Shoes", "Apparel", 100.0, 4.5),
    ("T-Shirt", "Apparel", 25.0, 4.0),
]
columns = ["product_name", "category", "price", "rating"]
df = spark.createDataFrame(data, columns)

# 3. Sort by Category ascending, then Price descending
sorted_products = df.orderBy(
    col("category").asc(),
    col("price").desc()
)

# 4. Sort by Rating descending to get top-rated products
top_rated = df.sort(col("rating").desc())

# 5. Show results
print("=== Sorted: Category (Asc), Price (Desc) ===")
sorted_products.show(truncate=False)

print("=== Top Rated Products (Rating Desc) ===")
top_rated.show(truncate=False)

Rendered Output:

=== Sorted: Category (Asc), Price (Desc) ===
+------------+-----------+------+------+
|product_name|category   |price |rating|
+------------+-----------+------+------+
|Running Shoes|Apparel    |100.0 |4.5   |
|T-Shirt     |Apparel    |25.0  |4.0   |
|Laptop      |Electronics|1200.0|4.8   |
|Smartphone  |Electronics|800.0 |4.6   |
|Headphones  |Electronics|150.0 |4.2   |
+------------+-----------+------+------+

=== Top Rated Products (Rating Desc) ===
+------------+-----------+------+------+
|product_name|category   |price |rating|
+------------+-----------+------+------+
|Laptop      |Electronics|1200.0|4.8   |
|Smartphone  |Electronics|800.0 |4.6   |
|Running Shoes|Apparel    |100.0 |4.5   |
|Headphones  |Electronics|150.0 |4.2   |
|T-Shirt     |Apparel    |25.0  |4.0   |
+------------+-----------+------+------+